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-13. (New) A method for determining an image shift in an image sequence to compensate for an 
image source movement, a plurality of image zones of images being available to determine the image 
shift, each of the plurality of image zones being at a specific position in the images and each having 
predefined dimensions of predefined numbers of pixels in different image directions, the method 
comprising: 

determining the image shift from one of (i) first image data of the first image and second 
image data of a second image, and (ii) the first image data of the first image and input image data of 
an input image for use in correcting the first image in the image sequence; 

determining a zone shift of any given image zone of the plurality of image zones from one of 
(i) the first image data of the first image and the second image data of the second image within the 
given image zone, and (ii) the first image data of the first image and the input image data of the input 
image within the given image zone, including determining the zone shift in two image zones and 
determining a reliability for the zone shift determination by: 

forming the zone shift and a correlation quotient for each of the two image zones; 
determining a threshold value function as a function of a corresponding value of 
determined zone shifts in the two image zones; 

comparing the correlation quotient of one of the two image zones to a comparison 
value obtained from the threshold value function for a zone shift of another of the two image 
zones; 

determining that the reliability of the zone shift determination is adequate for the one 
image zone of the two image zones if the correlation quotient determined for the one image 
zone is greater than the comparison value; and 

using the zone shift of one image zone of the plurality of image zones as the image shift as a 
function of the reliability for the zone shift determination of the one image zone. 

14. (New) The method of claim 13, wherein the correlation quotient for one of the plurality of 
image zones is determined by: 

determining shift correlation values for multiple possible zone shifts using block-matching; 

determining the zone shift of the plurality of image zones to be a zone shift associated with a 
maximum of the shift correlation values; and 
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forming the correlation quotient by dividing the maximum of the shift correlation values by an 
average of the shift correlation values. 

15. (New) The method of claim 13, wherein: 

the threshold value function assumes a predefined second threshold value for a given zone 
shift value less than a predefined first threshold value; 

the threshold value function assumes a value that is the predefined second threshold value 
minus a product for the given zone shift value greater than the predefined first threshold value; 

the product includes as factors a predefined gradient parameter and a difference; and 

the difference is formed from the given zone shift and the predefined first threshold value. 

16. (New) A method for determining an image shift in an image sequence of a plurality of images to 
compensate for a camera movement, at least one image zone of the plurality of images being 
available to determine the image shift, the at least one image zone being at a predefined position in 
the images and having predefined dimensions of predefined numbers of pixels in different image 
directions, the method comprising: 

determining the image shift from one of (i) first image data of a first image and second image 
data of a second image, and (ii) the first image data of the first image and input image data of an 
input image by using a zone shift of the at least one image zone as the image shift; and 

determining the zone shift using block shift information from block-based coding used for 
the at least one image zone, wherein: 

image blocks located in at least one image zone are reflected in the block shift 
information of the at least one image zone, and 

the at least one image zone is used to determine the image shift as a function of a 
reliability of a zone shift determination. 

17. (New) The method of claim 16, wherein the zone shift for the at least one image zone, the zone 
shift including a horizontal component and a vertical component, and the reliability of a zone shift 
determination are determined by: 

establishing a first frequency distribution of frequencies of different values for a horizontal 
component of the block shift information to determine the horizontal component of the zone shift, 
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the horizontal component of the zone shift corresponding to a horizontal component value of the 
block shift information for which the first frequency distribution assumes its primary maximum; 

establishing a second frequency distribution of frequencies of different values for a vertical 
component of the block shift information to determine the vertical component of the zone shift, the 
vertical component of the zone shift corresponding to a vertical component value of the of the block 
shift information for which the second frequency distribution assumes its primary maximum; 

determining that the reliability of the zone shift is adequate when the following conditions are 

met: 

an absolute value of a difference in position of values corresponding to the primary 
maximum and a secondary maximum of the first frequency distribution of the horizontal 
component of the block shift information is less than a predefined first difference threshold; 

the absolute value of the difference in position of the values corresponding to the 
primary maximum and a secondary maximum of the second frequency distribution of the 
vertical component of the block shift information is less than a predefined second difference 
threshold; 

the primary maximum of the first frequency distribution is greater than a first 
frequency threshold; and 

the primary maximum of the second frequency distribution is greater than a second 
frequency threshold. 

18. (New) The method of claim 13, further comprising: 

separating an image movement produced by the image source movement from an additional 
movement superimposed on the image movement in at least one of the image zones of the image to 
be corrected by: 

determining that a probability that the image movement will occur without the 
additional movement at different image positions; and 

determining the position and dimensions of a given image zone, and permanently 
specifying as a function of the probability that the image movement will occur without the 
additional movement within the given image zone. 

19. (New) The method of claim 13, further comprising: 
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selecting a position and dimensions of at least one first image zone so that the at least one 
first image zone of the images to be corrected is largely filled by an image background. 

20. (New) The method of claim 13, further comprising: 

selecting position and dimensions of at least one second image zone so that the at least one 
second image zone of the images to be corrected is largely filled with an image foreground. 

21 . (New) The method of claim 19, wherein the at least one first image zone and at least one 
second image zone are available to determine the image shift. 

22. (New) The method of claim 19, wherein two first image zones and a single second image zone 
are available for correcting the image, and the image shift is determined, in descending order of 
priority, from one of: 

an average of zone shifts of the first two image zones if the reliability of the zone shift 
determination of the first two image zones is determined to be adequate; 

the zone shift of one of the two first image zones for which the reliability of the zone shift 
determination is determined to be adequate; and 

a zone shift of the single second image zone. 

23. (New) The method of claim 22, wherein the source image movement is a camera movement 
and the method is used for a head-and-shoulder shot, further comprising: 

selecting the first two image zones in a side area to the left and right of a vertical center axis 
of a predefined rectangular image; and 

selecting the single second image zone in a center of the image with respect to the vertical 
center axis of the rectangular image; 

wherein: 

a first bottom distance of the first two image zones from a bottom of the image is 
greater than a first top distance of the first two image zones from a top of the image; and 

a second top distance of the single second image zone from the top of the image is 
greater than the second bottom distance of the single second image zone from the bottom of 
the image. 
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24. A device for determining an image shift, comprising: 

an image shift detecting arrangement, including a zone shift detector, an image storage 
device, and a microcomputer, wherein the shift detecting arrangement determines the image shift; 
and 

an enlarging arrangement; 

wherein the device is operable to determine the image shift in an image sequence to 
compensate for an image source movement, a plurality of image zones of images being available to 
determine the image shift, each of the plurality of image zones being at a specific position in the 
images and each having predefined dimensions of predefined numbers of pixels in different image 
directions, by: 

determining the image shift from one of (i) first image data of the first image and 
second image data of a second image, and (ii) the first image data of the first image and 
input image data of an input image for use in correcting the first image in the image 
sequence; 

determining a zone shift of any given image zone of the plurality of image zones from 
one of (i) the first image data of the first image and the second image data of the second 
image within the given image zone, and (ii) the first image data of the first image and the 
input image data of the input image within the given image zone, including determining the 
zone shift in two image zones and determining a reliability for the zone shift determination by: 
forming the zone shift and a correlation quotient for each of the two image 

zones; 

determining a threshold value function as a function of a corresponding 
value of determined zone shifts in the two image zones; 

comparing the correlation quotient of one of the two image zones to a 
comparison value obtained from the threshold value function for a zone shift of 
another of the two image zones; and 

determining that the reliability of the zone shift determination is adequate for 
the one image zone of the two image zones if the correlation quotient determined for 
the one image zone is greater than the comparison value; and 
using the zone shift of one image zone of the plurality of image zones as the image 
shift as a function of the reliability for the zone shift determination of the one image zone. 
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25. (New) The method of claim 13, wherein the image source is a camera. 

26. (New) The method of claim 13, wherein one of the following is satisfied: 

the second image data of the second image directly precedes the first image in the image 
sequence; and 

the input image data of the input image directly precedes the first image in the image 
sequence. 

27. (New) The method of claim 16, wherein one of the following is satisfied: 

the second image data of the second image directly precedes the first image in the image 
sequence; and 

the input image data of the input image directly precedes the first image in the image 
sequence. 

28. (New) The method of claim 16, wherein the block shift information includes image shift 
vectors. 

29. (New) The method of claim 18, wherein the image source is a camera. 

30. (New) The method of claim 18, wherein at least one first image zone is used to determine the 
image shift. 

3 1 . (New) The method of claim 13, further comprising: 

selecting a position and dimensions of at least one first image zone so that the at least one 
first image zone of the images to be corrected is largely filled by an image background; and 

selecting position and dimensions of at least one second image zone so that the at least one 
second image zone of the images to be corrected is largely filled with an image foreground. 

32. (New) The method of claim 31, wherein the at least one first image zone and the at least one 
second image zone are available to determine the image shift. 
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33. (New) The method of claim 20, wherein at least one first image zone and the at least one 
second image zone are available to determine the image shift. 

34. (New) The method of claim 23, wherein: 

the first two image zones is selected in a side area symmetric to the vertical center axis of 
the predefined rectangular image; and 

the single second image zone is selected in a center of the image symmetric to the vertical 
center axis of the rectangular image.—. 

Remarks 

This Preliminary Amendment cancels without prejudice original claims 1 to 12 in the 
underlying PCT Application No. PCT/DE00/00137, and adds without prejudice new claims 13 to 
34. The new claims conform the claims to U.S. Patent and Trademark Office rules and do not add 
new matter to the application. 

In accordance with 37 CJF.R. § 1.121(b)(3), the Substitute Specification (including 
the Abstract, but without the claims) contains no new matter. The amendments reflected in the 
Substitute Specification (including Abstract) are to conform the Specification and Abstract to U.S. 
Patent and Trademark Office rules or to correct informalities. As required by 37 C.F.R. § 
L121(b)(3)(iii) and § 1.125(b)(2), a Marked Up Version Of The Substitute Specification 
comparing the Specification of record and the Substitute Specification also accompanies this 
Preliminary Amendment. In the Marked Up Version, shading indicates added text and bracketing 
indicates deleted text. Approval and entry of the Substitute Specification (including Abstract) is 
respectfully requested. 

The underlying PCT Application No. PCT/DE00/00137 includes an International 
Search Report, dated June 23, 2000. The Search Report includes a list of documents that were 
uncovered in the underlying PCT Application. A copy of the Search Report accompanies this 
Preliminary Amendment. 

The underlying PCT Application No. PCT/DE00/00137 also includes an International 
Preliminary Examination Report, dated February 8, 2001 . An English translation of the 
International Preliminary Examination Report accompanies this Preliminary Amendment. 
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Applicant asserts that the subject matter of the present application is new, non- 
obvious, and useful. Prompt consideration and allowance of the application are respectfully 
requested. 



Dated: <?k '/Wl 




Gchard L. Mayer 
(Reg. No. 22,490) 

One Broadway 
New York, NY 10004 
(212) 425-7200 (telephone) 
(212) 425-5288 (facsimile) 




CUSTOMER NO. 26646 
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METHOD AND DEVICE FOR DETERMINING AN IMAGE SHIFT IN AN IMAGE 

SEQUENCE 



FIELD OF THE INVENTION 

The present invention relates to a method and device for 
determining an image shift in an image sequence. 

BACKGROUND INFORMATION 

A picture stabilization system whose function is to maximize 
the stability of a picture when shooting with a handheld 
camera is discussed in the reference "Bildstabilisation in 
Consumer-Camcordern, Funktion und Wirkungsweise" (Picture 
Stabilization in Consumer Camcorders: Function and Effects), 
H. Rindtorff, Fernseh- und Kinotechnik, vol. 49, no. 1/2, 
1995. As characterized, the image is divided into four image 
zones in which motion vectors that describe the image shift 
are determined. The motion vectors in the individual image 
zones purportedly yield an overall motion vector which ideally 
represents the camera travel. 

The entire motion vector is integrated into an attenuation 
factor, which means that past vector values are taken into 
account, and the magnitude of correction is reduced when the 
horizontal and vertical shifts exceed a limit value. 

SUMMARY OF THE INVENTION 

An exemplary method and/or exemplary embodiment of the present 
invention is directed to a method and/or device which provides 
that the reliability of image shift is not determined 
separately for the two image zones, but rather the determined 
shift in one of the two image zones determines the value 
assumed by the threshold value function and to which the shift 
of the second of the first two image zones are compared. 
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Another exemplary method of the present invention is directed 
to providing that the shift determination of one image zone 
and the determination of shift determination reliability thus 
do not take place independently of movements, for example, 
5 zone shifts, in other portions of an image. 

Another exemplary embodiment and/or exemplary method of the 
present invention is directed to determining the correlation 
quotient, in each case, for one of the plurality of image 
10 zones according to an exemplary method having the following 
steps : 

- shift correlation values are determined for multiple 
possible image shifts, using a block-matching method; 
■ : n - the image shift for which maximum shift correlation values 
15 are achieved is regarded as a shift in the image zone; 

,p - the correlation quotient is generated from the maximum of 
1% shift correlation values, divided by the average of the 

'"'4 determined shift correlation values. 



2© This exemplary embodiment and/or exemplary method of the 
if* present invention may be directed to determining the image 
Q shift through the entire image zone, since the shift 

correlation values are derived from a summation across all 
pixels or a selection of pixels within the image zone in 
25 question. Further, the correlation quotient may be normalized, 
since the maximum of the shift correlation values, from which 
the image shift is obtained, may be divided by the average of 
the calculated shift correlation values. 

30 Another exemplary embodiment and/or exemplary method of the 

present invention is directed to providing that for any given 
value of a zone shift that is less than a specified first 
threshold value, the threshold value function may assume the 
value of a predefined second threshold value; and for any 

35 given value of a zone shift that is greater than the 

predefined first threshold value, the threshold value function 
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may assume the value of the predefined second threshold value 
minus a product, the product including a predefined gradient 
parameter and a difference as factors, and the difference 
being formed from the given zone shift and the predefined 
5 first threshold value. Thus, an adequate reliability may be 
allowed for determining the zone shift in the case of larger 
determined shifts in an image zone, even if the correlation 
quotient is smaller, which may indicate a poorer correlation 
in the case of a smaller determined image shift. Due to the 

10 dependency of the first two image zones in determining 

reliability for the zone shift determination, the construction 
of the threshold value function means that the requirements 
imposed on the reliability of zone shift determination, i.e., 

; ; y the requirements imposed on the level of the correlation 

l;§ quotient may decrease in the use of a relatively large zone 

4" shift in one of the first two image zones. 

Another exemplary embodiment and/or exemplary method of the 
H; present invention is directed to providing that the method is 
2t) performed especially easily and economically, for example, the 
mi hardware support may allow obtainment of block shift 
I:: 1 information, for example, block shift vectors of small image 
blocks, at little or no additional cost, for example, from a 
block-based transmission method for reducing bandwidth. 

25 

Another exemplary embodiment and/or exemplary method of the 
present invention is directed to providing that for each of 
the image zones, an exemplary method having the following 
steps may be performed to determine a zone shift that includes 
30 a horizontal and a vertical component and to determine the 
reliability of the zone shift determination: 
- a first frequency distribution of the frequencies of 

different values of the horizontal component of the block 
shift information may be established to determine the 
35 horizontal component of the zone shift, with the horizontal 

component of the zone shift corresponding to the value of 
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the horizontal component of the block shift information for 
which the first frequency distribution may assume its 
primary maximum; 

- a second frequency distribution of the frequencies of 
different values of the vertical component of the block 
shift information may be established to determine the 
vertical component of the zone shift, with the vertical 
component of the zone shift corresponding to the value of 
the vertical component of the block shift information for 
which the second frequency distribution may assume its 
primary maximum; and 

- the reliability of the zone shift determination may be 
deemed to be adequate when all of the following conditions 
have been met: 

the absolute value of the difference between the 
position of the values, corresponding to the primary 
maximum and the secondary maximum of the first frequency 
distribution, of the horizontal component of the block 
shift information is less than a predefined first 
difference threshold; 

the absolute value of the difference between the 
position of the values, corresponding to the primary 
maximum and the secondary maximum of the second frequency 
distribution, of the vertical component of the block 
shift information is less than a predefined second 
difference threshold; 

the primary maximum of the first frequency 
distribution is greater than a first frequency threshold; 
and 

the primary maximum of the second frequency 
distribution is greater than a second frequency 
distribution . 

Using a relatively "simple" apparatus, arrangement, structure, 
or methods, such as, for example, monitoring the frequencies 
of the occurring horizontal and vertical components of the 
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existing block shift information, the zone shift of an image 
zone may be determined and the reliability of the zone shift 
determination may be determined. 

5 Another exemplary embodiment and/or exemplary method of the 

present invention is directed to separating an image movement, 
for example, produced by camera movement, from an additional 
movement that is superimposed on the image movement in some 
image zones of the image to be corrected, using the following 
10 steps: 

- the probability of an image movement occurring without the 
additional movement is determined at different positions in 
an image; 

i ; q - the position and dimensions are determined and permanently 

set for a given image zone as a function of the probability 
!: p that the image movement will occur within this one image 

1% zone without the additional movement; and 

:: 4 - at least one first image zone is given priority for use in 
j-, determining the shift. 

26 

in; In this way, image zones may be used for determining image 

shifts, for example, in those portions of an image that may be 
identified by a maximum probability that image movement will 
occur without the additional movement. The determination of 

25 image shifts in an image sequence may be performed with 

particular reliability. Another exemplary embodiment and/or 
exemplary method of the present invention is directed to 
permanently specifying the position and dimensions of the 
image zones for reliable determination of the image shifts in 

30 an image sequence. Thus, the shift may be reliably determined 
with relatively little processing effort, for example, for a 
special video communications scenario. A first image zone, 
which may be used to determine the shift, may therefore be 
employed by selecting its position and dimensions within the 

35 images, for example, solely to determine the shift, so that 

other image zones do not need to be taken into account in this 
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case, which may reduce the processing effort for determining 
the image shift. 

Another exemplary embodiment and/or exemplary method of the 
5 present invention is directed to providing that the position 
and dimensions of the at least one first image zone can be 
selected so that the at least one first image zone of the 
images to be corrected are largely filled by the image 
background. The image background may rarely contain any 
10 additional movement that is superimposed on the image movement 
produced, for example, by the unintentional movement of a 
camera, from one image to another between which the shift is 
to be determined, which can mean that a first image zone of 
: ;D the images to be corrected may be used to determine the shift, 
if provided that it is filled with the image background. 

Another exemplary embodiment and/or exemplary method of the 
: - present invention is directed to providing that the position 
O and dimensions of at least one second image zone may be 
|t selected so that the at least one second image zone of the 
1.0 images to be corrected is largely filled by the image 

;;7 foreground. Thus, a shift in the images of an image sequence 

may be determined, especially easily and accurately if, for 

example, the image background is subjected to strong 
25 additional movement that is superimposed on the image movement 

produced, for example, by the unintentional camera movement. 

Another exemplary embodiment and/or exemplary method of the 
present invention is directed to providing that both the at 

30 least one first image zone and the at least one second image 
zone are available for determining the shift. Thus, the zone 
shift determination may be checked because the exemplary 
method provides a measure for the reliability of the zone 
shift determination. A reliability determination for the 

35 reliability of the zone shift may serve primarily to easily 
and reliably separate the image movement from the additional 
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movement superimposed upon it. 

Another exemplary embodiment and/or exemplary method of the 
present invention is directed to providing that two first 
5 image zones and a single second image zone are available for 
image correction, with the shift being determined in at least 
one of the following: 

- from the average of the zone shifts of the two first image 
zones, when the reliability of the zone shift determination 

10 of the first two image sequences is deemed to be adequate; 

- from the zone shift of the one of the first two image zones 
in which the reliability of the zone shift determination is 
deemed to be adequate; and 

; ; o - from the zone shift of the second image zone. 

Different image zones having different priorities for 
■ri determining the image shifts in an image sequence may be used. 
u 4 For example, priority may be given to the use of image zones 
L. that are largely filled by the image background for 
26 determining the image shifts, with the use of the average of 
\n the zone shifts in the first two image zones yielding a 
□ reliable shift determination. The second priority for 

determining the shift, using the zone shift from the first two 
image zones, may be selected, for example, to minimize the 
25 influence of moving objects in the background, since a moving 
object in one of the two first image zones may mean that the 
zone shift in the other of the two image zones is used to 
determine the image shifts in an image sequence. On the third 
level of priority, the image may shift from the zone shift of 
30 the second image zone, for example, an image zone that is 
largely filled by the image background, may be used. 

Another exemplary embodiment and/or exemplary method of the 
present invention is directed to providing for a head-and- 
35 shoulder shot, with the first two image zones being selected 
in a side image zone on the left and right, for example, 
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symmetrically to the vertical center axis of a predefined 
rectangular image, with the distance of the first two image 
zones from the bottom of the image being greater than the 
distance of the first two image zones from the top of the 
5 image; the second image zone in the center of the image being 
selected, for example, symmetrically to the vertical center 
axis of the rectangular image; the distance of the second 
image zone from the top edge, for example, being greater than 
the distance of the second image zone from the bottom edge. 

10 The image zones may be selected so that, for example, in a 

head-and-shoulder shot, to rationally use information from the 
image zones for determining the zone shifts and for 
determining image shifts in an image sequence, based on a 

mi system of priorities. 

i: p Another exemplary embodiment and/or exemplary method of the 

present invention is directed to determining image shifts in 
'' :: 4 an image sequence and includes a shift detecting circuit (100) 
^ and an enlarging circuit (200), with the shift detecting 
26 circuit (100) including a zone shift detector (110), an image 
In storage device (120) and a microcomputer (130); and the shift 
detecting circuit (100) determining the shift (15). Thus, the 
shift may result in faster and more economical performance of 
an exemplary method of to the present invention, for example, 
25 by implementing the steps of the exemplary method of the 

present invention, for example, in an integrated circuit or on 
a personal computer (pc) board. 

BRIEF DESCRIPTION OF THE DRAWINGS 
30 Figure 1 shows a block diagram of an exemplary embodiment of 
the present invention. 

Figure 2 shows the picture stabilization system by determining 
an image shift in an image sequence. 

35 

Figure 3 shows a flowchart of the picture stabilization 
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system. 



Figure 4 shows selecting image zones within an image for 
performing an exemplary method of the present invention. 

5 

DETAILED DESCRIPTION 

Figure 1 shows a block diagram of an exemplary device of the 
present invention for implementing the exemplary method of 
determining an image shift in an image sequence. The exemplary 
10 device of the present invention may include an input 10, an 
output 20, a shift detecting circuit 100, and an enlarging 
circuit 200. Shift detecting circuit 100 may include zone 
shift detector 110, an image storage device 120, and a 
M3 microcomputer 130. Shift detecting circuit 100 may also 
l! include an input (not illustrated) that may be connected to 
;-C input 10 of the exemplary device of the v present invention as 
Q well as to zone shift detector 110 and image storage device 
120. Shift detecting circuit 100 may also include an output 
O (not illustrated) that may be connected to microcomputer 130, 

%t with zone shift detector 110 also being connected to 
m microcomputer 130. Enlarging circuit 200 may include two 

inputs (not illustrated) and one output that may be connected 
to output 20 of the device. The two inputs of enlarging 
circuit 200 may each be connected to an enlarging processor 
25 210, with one of the two inputs of enlarging circuit 200 being 
connected to input 10 of the device; and the other of the two 
inputs of enlarging circuit 200 being connected to the output 
of shift detecting circuit 100. 

30 Figure 2 shows an example of the picture stabilization system, 
for example, to compensate for camera movements. A first input 
image 13 may be corrected to form a first output image 23, 
using image information of a second, for example, earlier, 
input image 11. 



Second input image 11 includes an image segment that is 
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enlarged by increasing its size to form second output image 
21. Second input image 11 is completely picked up by the 
camera, although a user of an exemplary device of the present 
invention may see, for example, only the enlarged segment in 
the form of second output image 21. The segment may be 
referred to as second image 12 or as corrected second image 
12. 

According to a first exemplary method of the present 
invention, corrected second image 12 may be used to correct 
first input image 13. First input image 13 may also include an 
image segment that is referred to here as uncorrected first 
image 14. Comparing uncorrected first image 14 to second image 
12, i.e., corrected first image 12, allows for a determination 
of a shift 15 so that uncorrected first image 14 may be 
converted to a corrected first image 16 as a result of shift 
15. The comparison of uncorrected first image 14 to second 
image 12, for example, may not involve using all the image 
data, but rather only the image data from the image zones (not 
illustrated) of first image 14 and second image 12. 

According to a second exemplary method according to the 
present invention, second input image 11 may be used to 
correct first input image 13. Comparing first input image 13 
to second input image 11 may allow a determination of a shift 
15 so that uncorrected first image 14 may be converted to 
corrected first image 16 as a result of shift 15. The 
comparison of uncorrected first input image 13 to second input 
image 11, for example, may not involve using all the image 
data, but rather only the image data from the image zones (not 
illustrated) of first input image 13 and second input image 
11. 

According to both exemplary methods, corrected first image 16 
may now be displayed for the user in the form of first output 
image 23. Compared to the second output image, the shift of 
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first input image 13, i.e., corrected first image 16, 
respectively, may no longer be seen in first output image 23. 

Figure 3 shows steps of an exemplary method of the present 
5 invention, using the correction of first input image 13 as an 
example. The zone shifts of image zones may be determined in a 
first step 30. First input image 13 may be compared to 
corrected, for example, directly preceding second image 12 or 
to second, for example, directly preceding input for starting 

10 image 11. The shift of first image 14 may be subsequently 

determined in a step 40 in an exemplary method of the present 
invention. Uncorrected first image 14 may be shifted by shift 
vector 15 in a third step 50. This operation may yield 

; ; Q corrected first image 16. Corrected first image 16 may then be 

IS enlarged in a fourth step 60, resulting in first output image 

«p 23. 

To correct a further input image using an exemplary method 
according to the present invention, the result of third step 
£ r 6 50 may be supplied to first step 30 by storing the image, for 
in example, in image storage device 120. Third step 50 may yield 
corrected first image 16, which thus may replace corrected 
second image 12 and may be used in conjunction with the 
correction of the further input image to determine zone shifts 
25 in first step 30. 

Alternatively, first input image 13, i.e., uncorrected first 
image 14, may be stored in image storage device 120 together 
with determined shift 15 for correcting a further input image. 

30 

Figure 4 shows an example of a distribution of image segments 
6, 7, 8 within first input image 14. Two first image zones 6, 
7 may be selected, for example, for applying an exemplary 
method of the present invention to a head-and-shoulder shot, 
35 symmetrically to the vertical center axis of the given 

rectangular first image 14. The distance of first two image 
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zones 6,7 from the bottom of the image may be greater than the 
distance of first two image zones 6, 1 from the top of the 
image . 

5 A second image zone 8 may be selected in the center of first 
image 14, for example, symmetrically to the vertical center 
axis of the rectangular image, with the distance of second 
image zone 8 from the top of the image preferably being 
greater than the distance of the second image zone from the 
10 bottom of the image. 

An exemplary method of the present invention may be directed 
to determining a shift 15 of images in an image sequence and 

==S may be used for picture stabilization to compensate for camera 

1 : 5 : movements with regard to digital video stabilization in mobile 
video communications equipment. This may reduce and/or 

T~\ eliminate movements caused by the mobile use of video 

: ^ communications equipment. 

2t) The exemplary method may involve deriving the camera movement 
•■=:=••; from the relative shift in consecutive images and extracting 
E from one input image, e.g., first input image 13, the segment, 
for example, corrected first image 16, that compensates for 
the camera movement on the basis of determined shift 15, for 
25 example, of corrected first image 16 relative to uncorrected 
first image 14. 

With the exemplary method of the present invention, for 
example, a plurality of image zones 6, 7, 8 may be provided 

30 for determining shift 15. The image zones may be clearly 

determined by their positions and dimensions within the image. 
By selecting the position and dimensions of image zones 6, 7, 
8, it may be easy to separate an image movement that is 
produced by a camera movement from an additional movement that 

35 is superimposed on the image movement in segments of the image 
to be corrected. 
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For this purpose, the probability that the image movement will 
occur without the additional movement may be determined in 
different positions of the images in an image sequence, 
yielding preferred components within the image that can be 
5 used to separate the image movement from the additional 
movement. For example, selecting image zones 6, 7, 8 
illustrated in Figure 4 may be useful for the special shooting 
situation of a speaker which is centered in the image. The 
special characteristics of this shooting situation may be 
10 utilized as a-priori factors in selecting and defining image 
zones 6, 7 , 8 . 

According to this shooting situation, for example, the first 
two image zones 6, 7 are assumed to be largely located in the 

1 : 5" image background, while second image zone 8 is assumed to be 
largely in the foreground. This means that first two image 
zones 6, 7 may be primarily filled with image data from the 
image background, while second image zone 8 may be primarily 
filled with image data from the image foreground. This can 

20 allow a prioritization of first two image zones 6, 7, thereby 
^ determining a shift 15 of images in an image sequence, for 
example, by determining the zone shift in first two image 
zones 6, 7. Second image zone 8 may be used to determine image 
shift 15 only if the use of zone shifts from first two image 

25 zones 6, 7 allows for only a zone shift that is subject to 
high unreliability, i.e., to an insufficiently strong 
reliability. 

In the example given, this prioritization means that picture 
30 stabilization may be performed with image background 

information. However, the exemplary methods in this case do 
not apply exclusively to distinguishing and setting different 
priorities to determine a shift in images in an image sequence 
from background and foreground information, but also, for 
35 example, to using criteria such as edge determination, absence 
of edge determination and the like. 
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The first two image zones 6,7 in the example given may have a 
length of 120 pixels in the vertical direction and 40 pixels 
in the horizontal direction in Qcif format. In this format, 
second image zone 8 may measure 135 pixels in the vertical 
5 direction and 85 pixels in the horizontal direction. 

Another exemplary method of the present invention is directed 
to determining a shift 15 in images in an image sequence which 
also serves to minimize the influence of moving objects, for 

10 example, in the image background, by using a decision 

criterion to detect moving objects in image zones. The moving 
objects, for example, in one of image zones 6, 7, 8 may 
produce an additional movement that is superimposed on the 

!; n shift produced by the camera movement. 

i? 

:; p; To determine the zone shift for an image zone 6, 7, 8, two 

':ri alternative methods may be used according to the present 

i: 4 invention, depending on whether block shift information from a 

block-based coding method, for example, a block-based 
2% transmission method for reducing bandwidth, is accessible 
m using a relatively M simple" apparatus, arrangement, structure 
or method. If the block shift information is not easily 
accessible, a block-matching method may be used to determine 
the zone shift, making it possible to detect an additional 
25 movement, i.e., a local movement, within one of two first 

image zones 6, 7. A local movement occurring in an image zone 
6, 7, 8, e.g., an emerging object, may be detected by 
evaluating the shift correlation values from the block 
matching method. To do this, the ratio between the average of 
30 the shift correction values and the maximum of the correlation 
values may be compared with an adaptive threshold value 
function . 

To determine the average of the shift correlation values, the 
35 sum of all determined correlation values may be formed and 
subsequently divided by the number of these values. The 
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maximum of the shift correlation values may be assumed for a 
determined zone shift. The zone shift corresponding to the 
maximum of the shift correlation values may be assumed as the 
zone shift of the image zone. The correlation quotient may 
correspond to the maximum of shift correlation values, divided 
by the average of the shift correlation values, and may thus 
be normalized. An additional movement, i.e., a local movement 
within the image zone, may be detected if the correlation 
quotient is less than the value of an adaptive threshold value 
function. The adaptive threshold value function may be 
dependent on the length of a shift vector that indicates a 
zone shift. 

According to the exemplary embodiment and/or exemplary method 
of the present invention, the correlation quotient for first 
image zone 6, 7 may be compared to the value of the adaptive 
threshold value function to detect an additional movement, 
i.e., a local movement, in one of first two image zones 6, 7, 
the vector length of shift 15 of the other of first two image 
zones 6, 7 yielding the value for the adaptive threshold value 
function. The threshold value function may be defined as 
follows : 

- for any given vector length of the zone shift vector, i.e., 
for any given zone shift that is smaller than a predefined 
first threshold value, the threshold value function may 
assume a predefined second threshold value; 

- for any given vector length of the zone shift vector, i.e., 
for any given zone shift that is greater than the predefined 
first threshold value, the threshold value function may 
assume the value of the predefined second threshold value 
minus a product, with the product including a predefined 
gradient parameter and a difference as factors, and with the 
difference being derived from the given zone shift and the 
predefined first threshold value. 

Another exemplary method for determining image shifts in an 
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image sequence may be based, for example, on the use of block 
shift information from a block-based coding method. The shift 
vectors of small blocks, e.g., having a size of 8 x 8 or 16 x 
16, may be used to determine the zone shift in image zones 6, 
5 7, 8. The information from the block-based coding method may 
thus be used to reduce computing effort. This approach may be 
attractive if the block shift vectors need to be obtained with 
little or no additional effort, for example, using hardware 
support. Detecting local movement in an image zone 6, 7, 8, 
10 for example, in one of first two image zones 6, 7, may be 
accomplished with relative ease if shift vectors of small 
blocks in the image are available. 

!:□ Initially, all shift vectors for blocks located in one of 
r§ ; image zones 6, 7, 8 may be assigned to corresponding image 
"5 zone 6, 7, 8. Separate frequency distributions, i.e., 
■:j histograms, may be created from the assigned shift vectors for 
:= 4 the horizontal and vertical components. For each image zone 6, 
:=- ;i 7, 8, this may yield a first frequency distribution for the 
20 horizontal component and a second frequency distribution for 
iji the vertical component of the image block shift vectors. An 
y additional movement, i.e., local movement, may be detected by 
analyzing the frequency distributions assigned to an image 
zone. If the difference in position between the primary 
25 maximum and the largest secondary maximum of one of the two 
assigned frequency distributions exceeds a certain threshold 
value, and the size of the primary maximum drops below a 
threshold value, a local movement may have been detected. 
Detecting a local or additional movement in an image zone 
30 means that the zone shift may not have been determined with a 
sufficient amount of reliability. Determining the reliability 
of the zone shift determination thus may result in a negative 
result as far as zone shift determination is concerned. 



35 Another exemplary method of the present invention is directed 
to determining a zone shift and determining the reliability of 
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the zone shift determination and may be described as follows, 
with the zone shift including a horizontal and a vertical 
component : 

- a first frequency distribution of the frequencies of 

5 different values for the horizontal component of the block 

shift information is established to determine the horizontal 
component of the zone shift, with the horizontal component 
of the zone shift corresponding to the value of the 
horizontal component of the block shift information for 
10 which the first frequency distribution assumes its primary 

maximum; 

- a second frequency distribution of the frequencies of 
different values for the vertical component of the block 

hQ shift information is established to determine the vertical 

l ; g component of the zone shift, with the vertical component of 

the zone shift corresponding to the value of the vertical 
component of the block shift information for which the 
S J second frequency distribution assumes its primary maximum; 

H; - the reliability of the zone shift determination is deemed to 
|t be adequate when all of the following conditions have been 

Lfl met : 

llf - the absolute value of the difference in position of 

the values of the horizontal component of the block shift 
information, that correspond to the primary maximum and 

25 the secondary maximum of the first frequency 

distribution, is less than a predefined first difference 
threshold; 

the absolute value of the difference in position of 
the values of the vertical component of the block shift 
30 information, that correspond to the primary maximum and 

the secondary maximum of the second frequency 
distribution, is less than a predefined second difference 
threshold; 

the primary maximum of the first frequency 
35 distribution is greater than a first frequency threshold; 

and 
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the primary maximum of the second frequency 
distribution is greater than a second frequency 
threshold. 

The exemplary method may be used to determine image shifts in 
an image sequence, thus reducing computing effort. 

A criterion may be used that is suitable for detecting local 
movements within relevant image zone 6, 7, 8, so that the 
reliability of the zone shift determination is inadequate. 
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ABSTRACT OF THE DISCLOSURE 
A method and a device for determining an image shift in an 
image sequence may be used, for example, to compensate for a 
camera movement or other image source movement. A plurality of 
image zones of images are available to determine the shift, 
with the zone shift of any given image zone from a plurality 
of image zones being determined from the image data of, for 
example, consecutive images within any given image zone. A 
block matching method or a method using block shift 
information from a block-based coding method is used to 
determine the zone shift and to determine a reliability for 
the zone shift determination. 
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METHOD AND DEVICE FOR DETERMINING AN IMAGE SHIFT IN AN IMAGE 

SEQUENCE 

[Background Information 

1 FIELD OF THE INVENTION 

The present invention [is based on a method for determining an 
image shift in an image sequence according to the definition 
of species in the main claim and according to the definition 
of species in alternative independent Claim 4, as well as 
a] relates to a method and device for determining an image 
shift in an image sequence [ according to the definition of 
species in alternative independent Claim 12] . 

BACKGROUND INFORMATION 

A picture stabilization system whose function is to maximize 
the stability of a picture when shooting with a handheld 
camera is [known from] discussed in the [article by H. 
Rindtorff entitled] reference "Bildstabilisation in Consumer- 
Camcordern, Funktion und Wirkungsweise" (Picture Stabilization 
in Consumer Camcorders: Function and Effects) [ in], H. 
Rindtorff, Fernseh- und Kinotechnik, [Volume] vol. 49, [N]no. 
1/2, 1995. [According to this systemjAs characterized, the 
image is divided into four image zones in which motion vectors 
that describe the image shift are determined. The motion 
vectors in the individual image zones purportedly yield an 
overall motion vector which ideally represents the camera 
travel . 

The entire motion vector is integrated into an attenuation 
factor, which means that past vector values are taken into 
account, and the magnitude of correction is reduced when the 
horizontal and vertical shifts exceed a limit value. 
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Advantages of the Invention 



The method according to] 
SUMMARY 07 THE INVFXTIOK 

An exemplary method and/ or exemplary embodiment of the present 
invention [having the features of the main claim has the 
advantage over the related art] is directed to a method and/or 
device which provides that the reliability of image shift is 
not determined separately for the two image zones, but rather 
the determined shift in one of the two image zones determines 
the value assumed by the threshold value function and to which 
the shift of the second of the first two image zones are 
compared. [T] Another exempla-y method of the present invention 
is directed to providing that the shift determination of one 
image zone and the determination of shift determination 
reliability thus do not take place independently of movements, 
[in particular] for example, zone shifts, in other portions of 
an image. 

[The features mentioned in the subordinate claims provide 
advantageous embodiments and refinements of the method 
described in the main claim. 

One particular advantage is that] Another exemplary embodiment 
and/or exemplary method of the present invention is directed 
to determining the correlation quotient [ is determined], in 
each case, for one of the plurality of image zones according 
to [a] an exemplary method having the following steps: 

- shift correlation values are determined for multiple 
possible image shifts, using a block-matching method; 

- the image shift for which maximum shift correlation values 
are achieved is regarded as a shift in the image zone; 

- the correlation quotient is generated from the maximum of 
shift correlation values, divided by the average of the 
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determined shift correlation values. 



This [determines] exemplary embodiment and/or exemplary method 
of the present: invention may be directed to determining the 
image shift through the entire image zone, since the shift 
correlation values are derived from a summation across all 
pixels or a selection of pixels within the image zone in 
question. [In addition] Further , the correlation quotient 

[is] may be normalized, since the maximum of the shift 
correlation values, from which the image shift is obtained, 

[is] may be divided by the average of the calculated shift 

correlation values . 

Another [advantage is that ,] exemplary embodiment and/or 
exemplary method of the present invention is directed to 
providing that for any given value of a zone shift that is 
less than a specified first threshold value, the threshold 
value function may assume [s] the value of a predefined second 
threshold value; and for any given value of a zone shift that 
is greater than the predefined first threshold value, the 
threshold value function may assume [s] the value of the 
predefined second threshold value minus a product, the product 
including a predefined gradient parameter and a difference as 
factors, and the difference being formed from the given zone 
shift and the predefined first threshold value. [This has the 
advantage that] Thus, an adequate reliability [can]may be 
allowed for determining the zone shift in the case of larger 
determined shifts in an image zone, even if the correlation 
quotient is smaller, which [would] may indicate a poorer 
correlation in the case of a smaller determined image shift. 
Due to the dependency of the first two image zones in 
determining reliability for the zone shift determination, the 
construction of the threshold value function means that the 
requirements imposed on the reliability of zone shift 
determination, i.e., the requirements imposed on the level of 
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the correlation quotient [,] may decrease [s] in the use of a 
relatively large zone shift in one of the first two image 
zones . 

[The] Another exemplary embodiment and/or exemplary method 
[according to] of the present invention [having the features of 
alternative independent Claim 4 has the advantage over the 
re lated art that it enables the method according to the 
present invention to be carried out] is directed to providing 
that the method is performed especially easily and 
economically, [particularly if, ] f or example, the hardware 
support [makes it possible to obtain] may allow obtainment of 
block shift information, [in particular ] for example, block 
shift vectors of small image blocks, at little or no 
additional cost, [in particular] for example, from a block- 
based transmission method for reducing bandwidth. 

[The features mentioned in the subordinate claims provide 
advantageous embodiments and refinements of the method 
described in alternative independent Claim 4. 

One particular advantage is that ,] Another exemplary embodiment 
and/or exemplary method of the present invention is directed 
to providing that for each of the image zones, [a] an exemplary 
method having the following steps [is carried out] may be 
performed to determine a zone shift that includes a horizontal 
and a vertical component and to determine the reliability of 
the zone shift determination: 

- a first frequency distribution of the frequencies of 

different values of the horizontal component of the block 
shift information [is] may be established to determine the 
horizontal component of the zone shift, with the horizontal 
component of the zone shift corresponding to the value of 
the horizontal component of the block shift information for 
which the first frequency distribution may assume [s] its 
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primary maximum; 

- a second frequency distribution of the frequencies of 
different values of the vertical component of the block 
shift information [is] may be established to determine the 
vertical component of the zone shift, with the vertical 
component of the zone shift corresponding to the value of 
the vertical component of the block shift information for 
which the second frequency distribution may assume [s] its 
primary maximum; and 

- the reliability of the zone shift determination [is]may be 
deemed to be adequate when all of the following conditions 
have been met : 

the absolute value of the difference between the 
position of the values, corresponding to the primary 
maximum and the secondary maximum of the first frequency 
distribution, of the horizontal component of the block 
shift information is less than a predefined first 
difference threshold; 

the absolute value of the difference between the 
position of the values, corresponding to the primary 
maximum and the secondary maximum of the second frequency 
distribution, of the vertical component of the block 
shift information is less than a predefined second 
difference threshold; 

the primary maximum of the first frequency 
distribution is greater than a first frequency threshold; 
and 

the primary maximum of the second frequency 
distribution is greater than a second frequency 
distribution . 
[Using simple means, in particular] 

Using a relatively "simple" apparatus, arrangement, structure, 
or methods, such as, for example, monitoring the frequencies 
of the occurring horizontal and vertical components of the 
existing block shift information, [it is therefore possible to 
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determine ] the zone shift of an image zone [and to 
determine] may be determined and the reliability of the zone 
shift determination [ . 

5 According to the subordinate claims of both the main claim and 
alternative independent Claim 4, it is furthermore 
advantageous to separate] may be determined. 

Another exemplary embodiment and/or exemplary method of the 
10 present invention is directed to separating an image movement, 
[preferably one] for example, produced by camera movement, from 
an additional movement that is superimposed on the image 
movement in some image zones of the image to be corrected, 
~ l j using the following steps: 

IS - the probability of an image movement occurring without the 
"it additional movement is determined at different positions in 

an image; 

- the position and dimensions are determined and permanently 
■Z set for a given image zone as a function of the probability 
20 that the image movement will occur within this one image 

■ : P zone without the additional movement; and 

- at least one first image zone is given priority for use in 
determining the shift. 

25 [T]In this [makes it possible to use] way, image zones may be 

used for determining image shif ts [ particularly], for example, 
in those portions of an image that [are] may be identified by a 
maximum probability that image movement will occur without the 
additional movement. The determination of image shifts in an 

30 image sequence [can thus] may be [carried out ] performed with 
particular reliability. [It is also beneficial] Another 
exemplary embodiment and/or exemplary method of the present 
invention is directed to permanently [specif y] specif ying the 
position and dimensions of the image zones for reliable 

35 determination of the image shifts in an image sequence. Thus, 
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the shift [can] may be reliably determined with relatively 
little processing effort, [in particular ] for example, for a 
special video communications scenario. A first image zone, 
which [is preferably] may be used to determine the shift, 
5 [can] may therefore be employed by selecting its position and 

dimensions within the images, for example, solely to determine 
the shift, so that other image zones do not need to be taken 
into account in this case, which may reduce [s] the processing 
effort for determining the image shift. 

10 

Another [advantage is] exemplary embodiment and/or exemplary 
method of the present: invention is directed to providing that 
the position and dimensions of the at least one first image 
hi zone can be selected so that the at least one first image zone 

of the images to be corrected are largely filled by the image 
■,p background. The image background may rarely contain [s] any 

additional movement that is superimposed on the image movement 
"J produced, [in particular] for example, by the unintentional 
H; movement of a camera, from one image to another between which 
it) the shift is to be determined, which can mean[s] that a first 
ii--: image zone of the images to be corrected [can] may be [ 
□ advantageously] used to determine the shift, provided that it 
is filled with the image background. 

25 Another [advantage is] exemplary embodiment and/or exemplary 

method of the present invention is directed to providing that 
the position and dimensions of at least one second image zone 
[can] may be selected so that the at least one second image 
zone of the images to be corrected is largely filled by the 

30 image foreground. [This makes it possible to determine] Thus, a 
shift in the images of an image sequence may be determined, 
especially easily and accurately if, for example, the image 
background is subjected to strong additional movement that is 
superimposed on the image movement produced, for example, by 

35 the unintentional camera movement. 
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Another [advantage is] exemplary eir.bodiir.enr and/or exemplary 
method of che present invention is directed ~o providing that 
both the at least one first image zone and the at least one 
second image zone are available for determining the shift. 
[This makes it possible to check] Thus, the zone shift 
determination [, ] may be checked because the exemplary method 
provides a measure for the reliability of the zone shift 
determination. A reliability determination for the reliability 
of the zone shift [thus] may serve [s] primarily to easily and 
reliably separate the image movement from the additional 
movement superimposed upon it. 

[A further advantage is] Another exemplary embodiment and/or 
exemplary method of the present: invention is directed to 
providing that two first image zones and a single second image 
zone are available for image correction, with the shift being 
determined in az least one of the following [ three ways in 
descending order of priority] : 

- from the average of the zone shifts of the two first image 
zones, when the reliability of the zone shift determination 
of the first two image sequences is deemed to be adequate; 

- from the zone shift of the one of the first two image zones 
in which the reliability of the zone shift determination is 
deemed to be adequate; and 

- from the zone shift of the second image zone. 

[This makes it possible to use d] Different image zones having 
different priorities for determining the image shifts in an 
image sequence may be used. [In particular, it is possible to 
give]^or example, priority may be given to the use of image 
zones that are largely filled by the image background for 
determining the image shifts, with the use of the average of 
the zone shifts in the first two image zones yielding a 
reliable shift determination. The second priority for 
determining the shift, using the zone shift from the first two 
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image zones, [is] may be selected, [in particular] for example, 
to minimize the influence of moving objects in the background, 
since a moving object in one of the two first image zones may 
mean[s] that the zone shift in the other of the two image 
zones is used to determine the image shifts in an image 
sequence. On the third level of priority, the image iray 
shift [s] from the zone shift of the second image zone, [in 
particular] for example, an image zone that is largely filled 
by the image background, [is used. 

It is also advantageous to use the method] may be used. 

Another exemplary embodiment: and/or exemplary method of the 
present: invention is directed to providing for a head-and- 
shoulder shot, with the first two image zones being selected 
in a side image zone on the left and right, [preferably] for 
example, symmetrically to the vertical center axis of a 
predefined rectangular image, with the distance of the first 
two image zones from the bottom of the image being greater 
than the distance of the first two image zones from the top of 
the image; the second image zone in the center of the image 
being selected, [preferably] for example, symmetrically to the 
vertical center axis of the rectangular image; the distance of 
the second image zone from the top edge [ preferably], for 
example, being greater than the distance of the second image 
zone from the bottom edge. [Selecting t]The image zones 

[in]may be selected so th[is manner makes it possible] a:, for 
example, [ particularly] in a head-and-shoulder shot, to 
rationally use information from the image zones for 
determining the zone shifts and for determining image shifts 

in an image sequence, based on a system of priorities. 

[T]Ano:he[ device according to]r exemplary embodiment and/or 
exemplary method of the present invention [for] is directed to 
determining image shifts in an image sequence [having the 
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features of alternative independent Claim 12 has the advantage 
over the related art that the device] and includes a shift 
detecting circuit (100) and an enlarging circuit (200) r with 
the shift detecting circuit (100) including a zone shift 
detector (110) , an image storage device (120) and a 
microcomputer (130) ; and the shift detecting circuit (100) 
determining the shift (15) . Thus, the shift [can] may result in 
faster and more economical performance of [the] an exemplary 
method [according] of to the present invention, [in 
particular] for example,, by implementing the steps of the 
exemplary method [according to] of the present invention [ as 
described in the main claim and alternative independent Claim 
4], [respectively] for example, in an integrated circuit or on 
a [p.c. board. 

Brief Description of the Drawing 

One embodiment of the present invention is explained in 
greater detail in the following description and illustrated in 
the drawing, where. 

Figure 1 ] personal compute" 1 : (pc) board, 
BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a block diagram of [the device according to] an 
exemplary embodiment of the present invention [;] . 

Figure 2 [ ] shows [ the principle] the picture 

stabilization system by determining an image 
shift in an image sequence [;]. 

Figure 3 [ ] shows a flowchart of the picture stabilization 

system [ ; and] . 

Figure 4 [ ] shows [ an example for] selecting image zones 
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within an image for [carrying out 

the] performing an exemplary method [according 

to] of the present invention. 

r Description of the Embodiment 
1 DETAILED DESCRIPTION 

Figure 1 shows a block diagram of [the] an exemplary device 
[according to] of the present invention for Implementing rhe 
exemplary method of determining an image shift in an image 
sequence. The exemplary device [according to] of the present 
invention may include [s] an input 10, an output 20, a shift 
detecting circuit 100, and an enlarging circuit 200. Shift 
detecting circuit 100 may include [s] zone shift detector 110, 
an image storage device 120, and a microcomputer 130. Shift 
detecting circuit 100 may also include [s] an input (not 
illustrated) that [is] may be connected to input 10 of the 
exemplary device [according to] of the present invention as 
well as to zone shift detector 110 and image storage device 
120. Shift detecting circuit 100 [further] may also include [s] 
an output (not illustrated) that [is] may be connected to 
microcomputer 130, with zone shift detector 110 also being 
connected to microcomputer 130. Enlarging circuit 200 may 
include [s] two inputs (not illustrated) and one output that 
[is]may be connected to output 20 of the device. The two 
inputs of enlarging circuit 200 [are] may each be connected to 
an enlarging processor 210, with one of the two inputs of 
enlarging circuit 200 being connected to input 10 of the 
device; and the other of the two inputs of enlarging circuit 
200 being connected to the output of shift detecting circuit 
100. 

Figure 2 shows an example of the picture stabilization system, 
for example, to compensate for camera movements. A first input 
image 13 [is] may be corrected to form a first output image 23, 
using image information of a second, [preferably] for example, 
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earlier, input image 11. 



Second input image 11 includes an image segment that is 
enlarged by increasing its size to form second output image 
21. Second input image 11 is completely picked up by the 
camera, although a user of [the] an exemplary device [according 
to] of the present invention may see[s], for example, only the 
enlarged segment in the form of second output image 21. The 
segment [is] may be referred to as second image 12 or as 
corrected second image 12. 

According to a first [embodiment of the] exemplary method 
[according to] of the present invention, corrected second image 
12 [is] may be used to correct first input image 13. First 
input image 13 may also include [s] an image segment that is 
referred to here as uncorrected first image 14. Comparing 
uncorrected first image 14 to second image 12, i.e., corrected 
first image 12, [makes it possible to determine] allows for a 
determination of a shift 15 so that uncorrected first image 14 
[is]rr„ay be converted to a corrected first image 16 as a result 
of shift 15. The comparison of uncorrected first image 14 to 
second image 12, [in particular ] for example, [does]may not 
involve using all the image data, but rather only the image 
data from the image zones (not illustrated) of first image 14 
and second image 12. 

According to a second [embodiment of the] exemplary method 
according to the present invention, second input image 11 
[is] may be used to correct first input image 13. Comparing 
first input image 13 to second input image 11 [makes it 
possible to determine] may allow a determination of a shift 15 
so that uncorrected first image 14 [can] may be converted to 
corrected first image 16 as a result of shift 15. The 
comparison of uncorrected first input image 13 to second input 
image 11, [in particular] for example, [does] may not involve 
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using all the image data, but rather only the image data from 
the image zones (not illustrated) of first input image 13 and 
second input image 11. 

5 According to both [the first embodiment and the second 
embodiment of the method according to the present 
invention] exemplary methods, corrected first image 16 [can] may 
now be displayed for the user in the form of first output 
image 23. Compared to the second output image, the shift of 
10 first input image 13, i.e., corrected first image 16, 

respectively, [can] may no longer be seen in first output image 
23. 

■15 Figure 3 shows [the main j'steps [in the]cf an exemplary method 
|! [according to] of the present invention [ on the basis of the 
«P flowchart], using the correction of first input image 13 as an 
!|;' - example. The zone shifts of image zones [are] may be determined 

in a first step 30. First input image 13 [is]itay be compared 
O to corrected, [in particular] for example, directly preceding 
j>!t> second image 12 or to second, [in particular] for example, 
m directly preceding input for starring image 11. The shift of 
j£ first image 14 [is] may be subsequently determined in a step 40 
in [the] an exemplary method [according to] of the present 
invention. Uncorrected first image 14 [is] may be shifted by 
25 shift vector 15 in a third step 50. This operation may 

yield [s] corrected first image 16. Corrected first image 16 
[is] may then be enlarged in a fourth step 60, resulting in 
first output image 23. 

30 To correct a further input image using [the] an exemplary 

method according to the present invention, the result of third 
step 50 [can] may be supplied to first step 30 by storing the 
image, [preferably] for example, in image storage device 120. 
Third step 50 may yield [s] corrected first image 16, which 

35 thus may replace [s] corrected second image 12 and [is] may be 
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used in conjunction with the correction of the further input 
image to determine zone shifts in first step 30. 

Alternatively, first input image 13, i.e., uncorrected first 
5 image 14, [can] may be stored in image storage device 120 

together with determined shift 15 for correcting a further 
input image . 

Figure 4 shows an example of a distribution of image segments 
10 6, 7, 8 within first input image 14. Two first image zones 6, 
7 [are] may be selected, [in particular ] for example, for 
applying [the] an exemplary method [according to] of the present 
invention to a head-and-shoulder shot, symmetrically to the 
ijg vertical center axis of the given rectangular first image 14. 
15 The distance of first two image zones 6,7 from the bottom of 
the image [is]may be greater than the distance of first two 
image zones 6, 7 from the top of the image. 

;U; ; A second image zone 8 [is] may be selected in the center of 
it) first image 14, [preferably] for example, symmetrically to the 

vertical center axis of the rectangular image, with the 
□ distance of second image zone 8 from the top of the image 

preferably being greater than the distance of the second image 

zone from the bottom of the image. 

25 

[The] An exemplary method [according to] of the present 
invention [for] may be directed ':c determining a shift 15 of 
images in an image sequence [can preferably] and may be used 
for picture stabilization to compensate for camera movements 
30 with regard to digital video stabilization in mobile video 
communications equipment. [The goal is to] This may reduce 
[and, if possible, ] and/or eliminate movements caused by the 
mobile use of video communications equipment. 

35 The [basic principle behind the] exemplary method may i[s to 
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derive] nvolve deriving the camera movement from the relative 
shift in consecutive images and [to extract] ex-racking from 
one input image, e.g., first input image 13, the segment , for 
example, corrected first image 16, that compensates for the 
5 camera movement on the basis of determined shift 15, [in 

particular ] for example, of corrected first image 16 relative 
to uncorrected first image 14, 

With the exemplary method [according to] of the present 
10 invention, for example, a plurality of image zones 6, 7, 8 

[is] may be provided for determining shift 15. The image zones 

[can] may be clearly determined by their positions and 
:=«. dimensions within the image. By [advantageously ] selecting the 

position and dimensions of image zones 6, 7, 8, it [is 
1.5 especially] may be easy to separate an image movement that is[ 
: '7 : preferably] produced by a camera movement from an additional 
: u movement that is superimposed on the image movement in 

segments of the image to be corrected. 

2t) For this purpose, the probability that the image movement will 
■>y occur without the additional movement [is] may be determined in 
[?; different positions of the images in an image sequence, 

yielding preferred components within the image that can be 
used to separate the image movement from the additional 
25 movement. For example, selecting image zones 6, 7, 8 

illustrated in Figure 4 [is particularly advantageous] may be 
useful for the special shooting situation of a speaker which 
is centered in the image. The special characteristics of this 
shooting situation [arejmay be utilized as a-priori factors in 
30 selecting and defining image zones 6, 7, 8. 

According to this shooting situation, [we can assume] for 
example, th[at]e first two image zones 6, 7 are assumed :o be 
largely located in the image background, while second image 
35 zone 8 is assumed :o be largely in the foreground. This means 
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that first two image zones 6, 7 [are] may be primarily filled 
with image data from the image background, while second image 
zone 8 [is]may be primarily filled with image data from the 
image foreground. This [makes it possible to advantageously 
prioritize] can allow a prioritization of first two image zones 
6, 7 , thereby determining a shift 15 of images in an image 
sequence [ preferably], for example, by determining the zone 
shift in first two image zones 6, 7. Second image zone 8 
[is] may be used to determine image shift 15 only if the use of 
zone shifts from first two image zones 6, 7 allows for only a 
zone shift that is subject to high unreliability, i.e., to an 
insufficiently strong reliability. 

In the example given, this prioritization means that picture 
stabilization [is preferably carried out] may be performed with 
image background information. However, the exemplary methods 
in this case do not apply exclusively to distinguishing and 
setting different priorities to determine a shift in images in 
an image sequence from background and foreground information, 
but also, for example, to using criteria such as edge 
determination, absence of edge determination and the like. 

[F]The first two image zones 6,7 in the example given 
[typically]may have a length of 120 pixels in the vertical 
direction and 40 pixels in the horizontal direction in Qcif 
format. In this format, second image zone 8 [typically]may 
measure [s] 135 pixels in the vertical direction and 85 pixels 
in the horizontal direction. 

[The] Another exemplary method [according to] of the present 
invention [for] is directed to determining a shift 15 in images 
in an image sequence which also serves to minimize the 
influence of moving objects, [in particular] for example, in 
the image background, by using a decision criterion to detect 
moving objects in image zones. The moving objects, for 
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example, in one of image zones 6, 7, 8 may produce [s] an 
additional movement that is superimposed on the shift produced 
by the camera movement. 

To determine the zone shift for an image zone 6, 7, 8, two 
alternative methods [are] may be used according to the present 
invention, depending on whether [ or not] block shift 
information from a block-based coding method, for example, a 
block-based transmission method for reducing bandwidth, is 
accessible [with simple means] using a relatively "simple" 
apparatus, arrangement, siruccure or mechod. If the block 
shift information is not easily accessible, a block-matching 
method [according to the ]ma[in claim is preferably] y be used 
to determine the zone shift, making it possible to detect an 
additional movement, i.e., a local movement, within one of two 
first image zones 6, 7 . A local movement occurring in an image 
zone 6, 7, 8, e.g., an emerging object, [can] may be detected 
by evaluating the shift correlation values from the block 
matching method. To do this, the ratio between the average of 
the shift correction values and the maximum of the correlation 
values [is] may be compared with an adaptive threshold value 
function . 

To determine the average of the shift correlation values, the 
sum of all determined correlation values [is] may be formed and 
subsequently divided by the number of these values. The 
maximum of the shift correlation values [is] may be assumed for 
a determined zone shift. The zone shift corresponding to the 
maximum of the shift correlation values [is] may be assumed as 
the zone shift of the image zone. The correlation quotient may 
correspondfs] to the maximum of shift correlation values, 
divided by the average of the shift correlation values, and 
[is] may thus be normalized. An additional movement, i.e., a 
local movement within the image zone, [is]may be detected if 
the correlation quotient is less than the value of an adaptive 
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threshold value function. The adaptive threshold value 
function [is] may be dependent on the length of a shift vector 
that indicates a zone shift. [ ] 

5 According to the exemplary embodiment; and/or exemplary merhcd 
of the present invention, the correlation quotient for first 
image zone 6, 7 [in question is] may be compared to the value 
of the adaptive threshold value function to detect an 
additional movement, i.e., a local movement, in one of first 
10 two image zones 6, 7 , the vector length of shift 15 of the 

other of first two image zones 6, 7 yielding the value for the 
adaptive threshold value function. The threshold value 
^ function [is] tray be defined as follows: 

- for any given vector length of the zone shift vector, i.e., 
IS for any given zone shift that is smaller than a predefined 

^ first threshold value, the threshold value function may 

V assume [s] a predefined second threshold value; 

^ - for any given vector length of the zone shift vector, i.e., 

for any given zone shift that is greater than the predefined 
2t) first threshold value, the threshold value function may 

;f: assume [s] the value of the predefined second threshold value 

minus a product, with the product including a predefined 
gradient parameter and a difference as factors [;], and with 
the difference being derived from the given zone shift and 
25 the predefined first threshold value. 

[The] Another exemplary method for determining image shifts in 
an image sequence [according to alternative independent Claim 
4 is]may be based, [in particular ] for example, on the use of 

30 block shift information from a block-based coding method. The 
shift vectors of small blocks, e.g., having a size of 8 x 8 or 
16 x 16, [are] may be used to determine the zone shift in image 
zones 6, 7, 8. The information from the block-based coding 
method [is] may thus be used to reduce computing effort. This 

35 approach [is especially ] may be attractive if the block shift 
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vectors need to be obtained with little or no additional 
effort, for example, using hardware support. Detecting local 
movement in an image zone 6, 7, 8, [especially] for example, in 
one of first two image zones 6, 7 , [can]may be accomplished 
5 with relative ease if shift vectors of small blocks in the 
image are [ known] available . 

Initially, all shift vectors for blocks located in one of 
image zones 6, 7, 8 [are] may be assigned to corresponding 
10 image zone 6, 7, 8. Separate frequency distributions, i.e., 
histograms, [are] may be created from the assigned shift 
vectors for the horizontal and vertical components. For each 
^ image zone 6, 7, 8, this may yield [s] a first frequency 
hQ distribution for the horizontal component and a second 
iS frequency distribution for the vertical component of the image 
"P block shift vectors. An additional movement, i.e., local 
. movement, [is]may be detected by analyzing the frequency 
"4 distributions assigned to an image zone. If the difference in 
H; position between the primary maximum and the largest secondary 
|>t) maximum of one of the two assigned frequency distributions 
if exceeds a certain threshold value, and the size of the primary 

r maximum drops below a threshold value, a local movement may 

ha[s]ve been detected. 

Detecting a local or additional movement in an image zone 
25 means that the zone shift [could] may not have been determined 
with a sufficient amount of reliability. Determining the 
reliability of the zone shift determination thus may result [s] 
in a negative result as far as zone shift determination is 
concerned. 

30 

[The] Another exemplary method [according to alternative 
independent Claim 4 for] of rhe present: invention is directed 
to determining a zone shift and determining the reliability of 
the zone shift determination [can] and may be described as 
35 follows, with the zone shift including a horizontal and a 
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vertical component : 

- a first frequency distribution of the frequencies of 
different values for the horizontal component of the block 
shift information is established to determine the horizontal 
component of the zone shift, with the horizontal component 
of the zone shift corresponding to the value of the 
horizontal component of the block shift information for 
which the first frequency distribution assumes its primary 
maximum; 

- a second frequency distribution of the frequencies of 
different values for the vertical component of the block 
shift information is established to determine the vertical 
component of the zone shift, with the vertical component of 
the zone shift corresponding to the value of the vertical 
component of the block shift information for which the 
second frequency distribution assumes its primary maximum; 

- the reliability of the zone shift determination is deemed to 
be adequate when all of the following conditions have been 
met : 

the absolute value of the difference in position of 
the values of the horizontal component of the block shift 
information, that correspond to the primary maximum and 
the secondary maximum of the first frequency 
distribution, is less than a predefined first difference 
threshold; 

the absolute value of the difference in position of 
the values of the vertical component of the block shift 
information, that correspond to the primary maximum and 
the secondary maximum of the second frequency 
distribution, is less than a predefined second difference 
threshold; 

the primary maximum of the first frequency 
distribution is greater than a first frequency threshold; 

and 

the primary maximum of the second frequency 
MARKED UP VERSION OF THE SUBSTITUTE SPECIFICATION 



NY01 406651 v 1 



20 



distribution is greater than a second frequency 
threshold. 



The exemplary method [according to alternative independent 
5 Claim 4 can thus] may be used to determine image shifts in an 
image sequence, thus reducing computing effort. 

A criterion may be used that is suitable for detecting local 
movements within relevant image zone 6, 7, 8, [and which 
10 therefore means] so that the reliability of the zone shift 

determination is inadequate [, was given for both the method 
according to the main claim and the method according to 
alternative independent Claim 4. 

IS 
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Abstract 



Two methods are described for determining an image shift (15) 
in an image sequence and one] . 
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ABSTRACT OF THE DISCLOSURE 
A method and a device for determining an image shift [(15) ] in 
an image sequence [,] may be used, [in particular] for example, 
to compensate for a camera movement or c:her image source 
5 movement:. A plurality of image zones [ (6, 7, 8)] of images are 
available to determine the shift [ (15)], with the zone shift 
of any given image zone [(6, 7, 8) ] f rom a plurality of image 
zones [ (6, 7, 8)3 being determined from the image data of[ 
preferably], for example, consecutive images [ (14, 12)] within 
10 any given image zone [ (6, 7, 8); a]. A block matching method 
or a method using block shift information from a block-based 
coding method [being] is used to determine the zone shift and 
to determine a reliability for the zone shift determination. [ 



(Figure 3) ] 
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METHOD AND DEVICE FOR DETERMINING AN IMAGE SHIFT IN AN IMAGE 

SEQUENCE 



Background Information 



The present invention is based on a method for determining an 
image shift in an image sequence according to the definition 
5 of species in the main claim and according to the definition 
of species in alternative independent Claim 4, as well as a 
device for determining an image shift in an image sequence 
according to the definition of species in alternative 
'3 independent Claim 12. A picture stabilization system whose 
IS function is to maximize the stability of a picture when 

shooting with a handheld camera is known from the article by 
H. Rindtorff entitled "Bildstabilisation in Consumer- 
; :j Camcordern, Funktion und Wirkungsweise" (Picture Stabilization 

in Consumer Camcorders: Function and Effects) in Fernseh- und 
T5 Kinotechnik, Volume 49, No. 1/2, 1995. According to this 
M system, the image is divided into four image zones in which 

':]] motion vectors that describe the image shift are determined. 
:i : The motion vectors in the individual image zones yield an 
overall motion vector which ideally represents the camera 
20 travel. 



The entire motion vector is integrated into an attenuation 
factor, which means that past vector values are taken into 
account, and the magnitude of correction is reduced when the 
25 horizontal and vertical shifts exceed a limit value. 



Advantages of the Invention 



The method according to the present invention having the 
30 features of the main claim has the advantage over the related 
art that the reliability of image shift is not determined 
separately for the two image zones, but rather the determined 
shift in one of the two image zones determines the value 
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assumed by the threshold value function and to which the shift 
of the second of the first two image zones are compared. The 
shift determination of one image zone and the determination of 
shift determination reliability thus do not take place 
independently of movements, in particular zone shifts, in 
other portions of an image. 

The features mentioned in the subordinate claims provide 
advantageous embodiments and refinements of the method 
described in the main claim. 

One particular advantage is that the correlation quotient is 
determined, in each case, for one of the plurality of image 
zones according to a method having the following steps: 

- shift correlation values are determined for multiple 
possible image shifts, using a block-matching method; 

- the image shift for which maximum shift correlation values 
are achieved is regarded as a shift in the image zone; 

- the correlation quotient is generated from the maximum of 
shift correlation values, divided by the average of the 
determined shift correlation values. 

This determines the image shift through the entire image zone, 
since the shift correlation values are derived from a 
summation across all pixels or a selection of pixels within 
the image zone in question. In addition, the correlation 
quotient is normalized, since the maximum of the shift 
correlation values, from which the image shift is obtained, is 
divided by the average of the calculated shift correlation 
values . 

Another advantage is that, for any given value of a zone shift 
that is less than a specified first threshold value, the 
threshold value function assumes the value of a predefined 
second threshold value; and for any given value of a zone 
shift that is greater than the predefined first threshold 
value, the threshold value function assumes the value of the 
predefined second threshold value minus a product, the product 
including a predefined gradient parameter and a difference as 
factors, and the difference being formed from the given zone 
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shift and the predefined first threshold value. This has the 
advantage that an adequate reliability can be allowed for 
determining the zone shift in the case of larger determined 
shifts in an image zone, even if the correlation quotient is 
smaller, which would indicate a poorer correlation in the case 
of a smaller determined image shift. Due to the dependency of 
the first two image zones in determining reliability for the 
zone shift determination, the construction of the threshold 
value function means that the requirements imposed on the 
reliability of zone shift determination, i.e., the 
requirements imposed on the level of the correlation quotient, 
decreases in the use of a relatively large zone shift in one 
of the first two image zones. 

The method according to the present invention having the 
features of alternative independent Claim 4 has the advantage 
over the related art that it enables the method according to 
the present invention to be carried out especially easily and 
economically, particularly if, for example, hardware support 
makes it possible to obtain block shift information, in 
particular block shift vectors of small image blocks, at 
little or no additional cost, in particular from a block-based 
transmission method for reducing bandwidth. 

The features mentioned in the subordinate claims provide 
advantageous embodiments and refinements of the method 
described in alternative independent Claim 4. 

One particular advantage is that, for each of the image zones, 
a method having the following steps is carried out to 
determine a zone shift that includes a horizontal and a 
vertical component and to determine the reliability of the 
zone shift determination: 

- a first frequency distribution of the frequencies of 

different values of the horizontal component of the block 
shift information is established to determine the horizontal 
component of the zone shift, with the horizontal component 
of the zone shift corresponding to the value of the 
horizontal component of the block shift information for 



NY01 402040 v 1 



3 



which the first frequency distribution assumes its primary 
maximum; 

- a second frequency distribution of the frequencies of 
different values of the vertical component of the block 
5 shift information is established to determine the vertical 

component of the zone shift, with the vertical component of 
the zone shift corresponding to the value of the vertical 
component of the block shift information for which the 
second frequency distribution assumes its primary maximum; 
10 - the reliability of the zone shift determination is deemed to 
be adequate when all of the following conditions have been 
met : 

- the absolute value of the difference between the position 
of the values, corresponding to the primary maximum and 

X|J the secondary maximum of the first frequency 

; J3 distribution, of the horizontal component of the block 

'Z shift information is less than a predefined first 

difference threshold; 

- the absolute value of the difference between the position 
20 of the values, corresponding to the primary maximum and 

! :J the secondary maximum of the second frequency 

U distribution, of the vertical component of the block 

shift information is less than a predefined second 
}il difference threshold; 

25 - the primary maximum of the first frequency distribution 

is greater than a first frequency threshold; 

- the primary maximum of the second frequency distribution 
is greater than a second frequency distribution. 

Using simple means, in particular monitoring the frequencies 
30 of the occurring horizontal and vertical components of the 

existing block shift information, it is therefore possible to 
determine the zone shift of an image zone and to determine the 
reliability of the zone shift determination. 

35 According to the subordinate claims of both the main claim and 
alternative independent Claim 4, it is furthermore 
advantageous to separate an image movement, preferably one 
produced by camera movement, from an additional movement that 
is superimposed on the image movement in some image zones of 



NY01 402040 v 1 



4 



the image to be corrected, using the following steps: 

- the probability of an image movement occurring without the 
additional movement is determined at different positions in 
an image ; 

- the position and dimensions are determined and permanently 
set for a given image zone as a function of the probability 
that the image movement will occur within this one image 
zone without the additional movement; 

- at least one first image zone is given priority for use in 
determining the shift. 

This makes it possible to use image zones for determining 
image shifts particularly in those portions of an image that 
are identified by a maximum probability that image movement 
will occur without the additional movement. The determination 
of image shifts in an image sequence can thus be carried out 
with particular reliability. It is also beneficial to 
permanently specify the position and dimensions of the image 
zones for reliable determination of the image shifts in an 
image sequence. Thus, the shift can be reliably determined 
with relatively little processing effort, in particular for a 
special video communications scenario. A first image zone, 
which is preferably used to determine the shift, can therefore 
be employed by selecting its position and dimensions within 
the images, for example solely to determine the shift, so that 
other image zones do not need to be taken into account in this 
case, which reduces the processing effort for determining the 
image shift. 

Another advantage is that the position and dimensions of the 
at least one first image zone can be selected so that the at 
least one first image zone of the images to be corrected are 
largely filled by the image background. The image background 
rarely contains any additional movement that is superimposed 
on the image movement produced, in particular, by the 
unintentional movement of a camera, from one image to another 
between which the shift is to be determined, which means that 
a first image zone of the images to be corrected can be 
advantageously used to determine the shift, provided that it 
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is filled with the image background. 

Another advantage is that the position and dimensions of at 
least one second image zone can be selected so that the at 
least one second image zone of the images to be corrected is 
largely filled by the image foreground. This makes it possible 
to determine a shift in the images of an image sequence 
especially easily and accurately if, for example, the image 
background is subjected to strong additional movement that is 
superimposed on the image movement produced, for example, by 
the unintentional camera movement. 

Another advantage is that both the at least one first image 
zone and the at least one second image zone are available for 
determining the shift. This makes it possible to check the 
zone shift determination, because the method provides a 
measure for the reliability of the zone shift determination. A 
reliability determination for the reliability of the zone 
shift thus serves primarily to easily and reliably separate 
the image movement from the additional movement superimposed 
upon it . 

A further advantage is that two first image zones and a single 
second image zone are available for image correction, with the 
shift being determined in one of the following three ways in 
descending order of priority: 

- from the average of the zone shifts of the two first image 
zones, when the reliability of the zone shift determination 
of the first two image sequences is deemed to be adequate; 

- from the zone shift of the one of the first two image zones 
in which the reliability of the zone shift determination is 
deemed to be adequate; 

- from the zone shift of the second image zone. 

This makes it possible to use different image zones having 
different priorities for determining the image shifts in an 
image sequence. In particular, it is possible to give priority 
to the use of image zones that are largely filled by the image 
background for determining the image shifts, with the use of 
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the average of the zone shifts in the first two image zones 
yielding a reliable shift determination. The second priority 
for determining the shift, using the zone shift from the first 
two image zones, is selected, in particular, to minimize the 
influence of moving objects in the background, since a moving 
object in one of the two first image zones means that the zone 
shift in the other of the two image zones is used to determine 
the image shifts in an image sequence. On the third level of 
priority, the image shifts from the zone shift of the second 
image zone, in particular an image zone that is largely filled 
by the image background, is used. 

It is also advantageous to use the method for a head- and - 
shoulder shot, with the first two image zones being selected 
in a side image zone on the left and right, preferably 
symmetrically to the vertical center axis of a predefined 
rectangular image, with the distance of the first two image 
zones from the bottom of the image being greater than the 
distance of the first two image zones from the top of the 
image; the second image zone in the center of the image being 
selected, preferably symmetrically to the vertical center axis 
of the rectangular image; the distance of the second image 
zone from the top edge preferably being greater than the 
distance of the second image zone from the bottom edge. 
Selecting the image zones in this manner makes it possible, 
particularly in a head-and-shoulder shot, to rationally use 
information from the image zones for determining the zone 
shifts and for determining image shifts in an image sequence, 
based on a system of priorities. 

The device according to the present invention for determining 
image shifts in an image sequence having the features of 
alternative independent Claim 12 has the advantage over the 
related art that the device includes a shift detecting circuit 
(100) and an enlarging circuit (200) , with the shift detecting 
circuit (100) including a zone shift detector (110) , an image 
storage device (12 0) and a microcomputer (130) ,- and the shift 
detecting circuit (100) determining the shift (15) . Thus, the 
shift can result in faster and more economical performance of 
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the method according to the present invention, in particular 
by implementing the steps of the method according to the 
present invention as described in the main claim and 
alternative independent Claim 4, respectively, in an 
integrated circuit or on a p. c. board. 

Brief Description of the Drawing 

One embodiment of the present invention is explained in 
greater detail in the following description and illustrated in 
the drawing , where . 

Figure 1 shows a block diagram of the device according to the 
present invention; 

Figure 2 shows the principle the picture stabilization system 
by determining an image shift in an image sequence; 

Figure 3 shows a flowchart of the picture stabilization 
system; and 

Figure 4 shows an example for selecting image zones within an 
image for carrying out the method according to the 
present invention . 

Description of the Embodiment 

Figure 1 shows a block diagram of the device according to the 
present invention for determining an image shift in an image 
sequence. The device according to the present invention 
includes an input 10, an output 20, a shift detecting circuit 
100, and an enlarging circuit 200. Shift detecting circuit 100 
includes zone shift detector 110, an image storage device 120, 
and a microcomputer 130. Shift detecting circuit 100 also 
includes an input (not illustrated) that is connected to input 
10 of the device according to the present invention as well as 
to zone shift detector 110 and image storage device 120. Shift 
detecting circuit 100 further includes an output (not 
illustrated) that is connected to microcomputer 13 0, with zone 
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shift detector 110 also being connected to microcomputer 130. 
Enlarging circuit 200 includes two inputs (not illustrated) 
and one output that is connected to output 20 of the device. 
The two inputs of enlarging circuit 200 are each connected to 
an enlarging processor 210, with one of the two inputs of 
enlarging circuit 200 being connected to input 10 of the 
device; and the other of the two inputs of enlarging circuit 
200 being connected to the output of shift detecting circuit 
100 . 

Figure 2 shows an example of the picture stabilization system, 
for example, to compensate for camera movements. A first input 
image 13 is corrected to form a first output image 23, using 
image information of a second, preferably earlier, input image 
11 . 

Second input image 11 includes an image segment that is 
enlarged by increasing its size to form second output image 
21. Second input image 11 is completely picked up by the 
camera, although a user of the device according to the present 
invention sees, for example, only the enlarged segment in the 
form of second output image 21. The segment is referred to as 
second image 12 or as corrected second image 12 . 

According to a first embodiment of the method according to the 
present invention, corrected second image 12 is used to 
correct first input image 13. First input image 13 also 
includes an image segment that is referred to here as 
uncorrected first image 14. Comparing uncorrected first image 
14 to second image 12, i.e., corrected first image 12, makes 
it possible to determine a shift 15 so that uncorrected first 
image 14 is converted to a corrected first image 16 as a 
result of shift 15. The comparison of uncorrected first image 
14 to second image 12, in particular, does not involve using 
all the image data, but rather only the image data from the 
image zones (not illustrated) of first image 14 and second 
image 12 . 

According to a second embodiment of the method according to 
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the present invention, second input image 11 is used to 
correct first input image 13 . Comparing first input image 13 
to second input image 11 makes it possible to determine a 
shift 15 so that uncorrected first image 14 can be converted 
to corrected first image 16 as a result of shift 15. The 
comparison of uncorrected first input image 13 to second input 
image 11, in particular, does not involve using all the image 
data, but rather only the image data from the image zones (not 
illustrated) of first input image 13 and second input image 
11 . 

According to both the first embodiment and the second 
embodiment of the method according to the present invention, 
corrected first image 16 can now be displayed for the user in 
the form of first output image 23. Compared to the second 
output image, the shift of first input image 13, i.e., 
corrected first image 16, respectively, can no longer be seen 
in first output image 23. 

Figure 3 shows the main steps in the method according to the 
present invention on the basis of the flowchart, using the 
correction of first input image 13 as an example. The zone 
shifts of image zones are determined in a first step 30. First 
input image 13 is compared to corrected, in particular, 
directly preceding second image 12 or to second, in 
particular, directly preceding input image 11. The shift of 
first image 14 is subsequently determined in a step 40 in the 
method according to the present invention. Uncorrected first 
image 14 is shifted by shift vector 15 in a third step 50. 
This operation yields corrected first image 16. Corrected 
first image 16 is then enlarged in a fourth step 60, resulting 
in first output image 23. 

To correct a further input image using the method according to 
the present invention, the result of third step 50 can be 
supplied to first step 3 0 by storing the image, preferably in 
image storage device 120. Third step 50 yields corrected first 
image 16, which thus replaces corrected second image 12 and is 
used in conjunction with the correction of the further input 
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image to determine zone shifts in first step 30. 

Alternatively, first input image 13, i.e., uncorrected first 
image 14, can be stored in image storage device 12 0 together 
with determined shift 15 for correcting a further input image. 

Figure 4 shows an example of a distribution of image segments 
6, 7, 8 within first input image 14. Two first image zones 6, 

7 are selected, in particular, for applying the method 
according to the present invention to a head- and- shoulder 
shot, symmetrically to the vertical center axis of the given 
rectangular first image 14. The distance of first two image 
zones 6,7 from the bottom of the image is greater than the 
distance of first two image zones 6, 7 from the top of the 
image . 

A second image zone 8 is selected in the center of first image 
14, preferably symmetrically to the vertical center axis of 
the rectangular image, with the distance of second image zone 

8 from the top of the image preferably being greater than the 
distance of the second image zone from the bottom of the 
image . 

The method according to the present invention for determining 
a shift 15 of images in an image sequence can preferably be 
used for picture stabilization to compensate for camera 
movements with regard to digital video stabilization in mobile 
video communications equipment. The goal is to reduce and, if 
possible, eliminate movements caused by the mobile use of 
video communications equipment. 

The basic principle behind the method is to derive the camera 
movement from the relative shift in consecutive images and to 
extract from one input image, e.g., first input image 13, the 
segment, for example corrected first image 16, that 
compensates for the camera movement on the basis of determined 
shift 15, in particular, of corrected first image 16 relative 
to uncorrected first image 14. 

With the method according to the present invention, for 
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example, a plurality of image zones 6, 7, 8 is provided for 
determining shift 15. The image zones can be clearly 
determined by their positions and dimensions within the image. 
By advantageously selecting the position and dimensions of 
5 image zones 6, 7, 8, it is especially easy to separate an 
image movement that is preferably produced by a camera 
movement from an additional movement that is superimposed on 
the image movement in segments of the image to be corrected. 
For this purpose, the probability that the image movement will 

10 occur without the additional movement is determined in 
different positions of the images in an image sequence, 
yielding preferred components within the image that can be 
used to separate the image movement from the additional 
movement. For example, selecting image zones 6, 7, 8 

IS illustrated in Figure 4 is particularly advantageous for the 
special shooting situation of a speaker which is centered in 
• the image. The special characteristics of this shooting 

. w situation are utilized as a-priori factors in selecting and 
defining image zones 6, 7, 8. According to this shooting 

20 situation, we can assume that first two image zones 6, 7 are 
•~ largely located in the image background, while second image 
; - zone 8 is largely in the foreground. This means that first two 

~'~J image zones 6, 7 are primarily filled with image data from the 

•I; image background, while second image zone 8 is primarily 

25 filled with image data from the image foreground. This makes 

it possible to advantageously prioritize first two image zones 
6, 7, thereby determining a shift 15 of images in an image 
sequence preferably by determining the zone shift in first two 
image zones 6, 7. Second image zone 8 is used to determine 

30 image shift 15 only if the use of zone shifts from first two 
image zones 6, 7 allows for only a zone shift that is subject 
to high unreliability, i.e., to an insufficiently strong 
reliability. In the example given, this prioritization means 
that picture stabilization is preferably carried out with 

35 image background information. However, the methods in this 
case do not apply exclusively to distinguishing and setting 
different priorities to determine a shift in images in an 
image sequence from background and foreground information, but 
also, for example, to using criteria such as edge 
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determination, absence of edge determination and the like. 

First two image zones 6,7 in the example given typically have 
a length of 120 pixels in the vertical direction and 40 pixels 
5 in the horizontal direction in Qcif format. In this format, 
second image zone 8 typically measures 13 5 pixels in the 
vertical direction and 85 pixels in the horizontal direction. 

The method according to the present invention for determining 
10 a shift 15 in images in an image sequence also serves to 

minimize the influence of moving objects, in particular in the 
image background, by using a decision criterion to detect 
moving objects in image zones. The moving objects, for 
example, in one of image zones 6, 7, 8 produces an additional 
l$j movement that is superimposed on the shift produced by the 
camera movement . 

To determine the zone shift for an image zone 6, 7, 8, two 
j; alternative methods are used according to the present 
2 ; 0 invention, depending on whether or not block shift information 
H_ from a block-based coding method, for example a block-based 
M transmission method for reducing bandwidth, is accessible with 

simple means. If the block shift information is not easily 
; T- accessible, a block-matching method according to the main 
25 claim is preferably used to determine the zone shift, making 
it possible to detect an additional movement, i.e., a local 
movement, within one of two first image zones 6, 7. A local 
movement occurring in an image zone 6, 7, 8, e.g., an emerging 
object, can be detected by evaluating the shift correlation 
30 values from the block matching method. To do this, the ratio 
between the average of the shift correction values and the 
maximum of the correlation values is compared with an adaptive 
threshold value function. To determine the average of the 
shift correlation values, the sum of all determined 
35 correlation values is formed and subsequently divided by the 
number of these values. The maximum of the shift correlation 
values is assumed for a determined zone shift. The zone shift 
corresponding to the maximum of the shift correlation values 
is assumed as the zone shift of the image zone. The 
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correlation quotient corresponds to the maximum of shift 
correlation values f divided by the average of the shift 
correlation values, and is thus normalized. An additional 
movement, i.e., a local movement within the image zone, is 
5 detected if the correlation quotient is less than the value of 
an adaptive threshold value function. The adaptive threshold 
value function is dependent on the length of a shift vector 
that indicates a zone shift. According to the present 
invention, the correlation quotient for first image zone 6, 7 
10 in question is compared to the value of the adaptive threshold 
value function to detect an additional movement, i.e., a local 
movement, in one of first two image zones 6, 7, the vector 
length of shift 15 of the other of first two image zones 6, 7 
yielding the value for the adaptive threshold value function. 
1-3 The threshold value function is defined as follows: 

- for any given vector length of the zone shift vector, i.e., 
X. for any given zone shift that is smaller than a predefined 

X first threshold value, the threshold value function assumes 

a predefined second threshold value; 
20 - for any given vector length of the zone shift vector, i.e., 

for any given zone shift that is greater than the predefined 
first threshold value, the threshold value function assumes 
the value of the predefined second threshold value minus a 
product, with the product including a predefined gradient 
25 parameter and a difference as factors; and with the 

difference being derived from the given zone shift and the 
predefined first threshold value. 

The method for determining image shifts in an image sequence 
30 according to alternative independent Claim 4 is based, in 
particular, on the use of block shift information from a 
block-based coding method. The shift vectors of small blocks, 
e.g., having a size of 8 x 8 or 16 x 16, are used to determine 
the zone shift in image zones 6, 7, 8. The information from 
35 the block-based coding method is thus used to reduce computing 
effort. This approach is especially attractive if the block 
shift vectors need to be obtained with little or no additional 
effort, for example using hardware support. Detecting local 
movement in an image zone 6, 7, 8, especially in one of first 
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two image zones 6, 7, can be accomplished with relative ease 
if shift vectors of small blocks in the image are known. 
Initially, all shift vectors for blocks located in one of 
image zones 6, 7, 8 are assigned to corresponding image zone 
5 6, 7, 8. Separate frequency distributions, i.e., histograms, 

are created from the assigned shift vectors for the horizontal 
and vertical components. For each image zone 6, 7, 8, this 
yields a first frequency distribution for the horizontal 
component and a second frequency distribution for the vertical 

10 component of the image block shift vectors. An additional 

movement, i.e., local movement, is detected by analyzing the 
frequency distributions assigned to an image zone. If the 
difference in position between the primary maximum and the 
largest secondary maximum of one of the two assigned frequency 

15i- distributions exceeds a certain threshold value, and the size 
: " v of the primary maximum drops below a threshold value, a local 
: /: movement has been detected. 

Detecting a local or additional movement in an image zone 
means that the zone shift could not be determined with a 

20 sufficient amount of reliability. Determining the reliability 

^ of the zone shift determination thus results in a negative 
result as far as zone shift determination is concerned. 

The method according to alternative independent Claim 4 for 
25 determining a zone shift and determining the reliability of 

the zone shift determination can be described as follows, with 
the zone shift including a horizontal and a vertical 
component : 

- a first frequency distribution of the frequencies of 

30 different values for the horizontal component of the block 

shift information is established to determine the horizontal 
component of the zone shift, with the horizontal component 
of the zone shift corresponding to the value of the 
horizontal component of the block shift information for 

35 which the first frequency distribution assumes its primary 

maximum; 

- a second frequency distribution of the frequencies of 
different values for the vertical component of the block 
shift information is established to determine the vertical 
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component of the zone shift, with the vertical component of 
the zone shift corresponding to the value of the vertical 
component of the block shift information for which the 
second frequency distribution assumes its primary maximum; 
5 - the reliability of the zone shift determination is deemed to 
be adequate when all of the following conditions have been 
met : 

- the absolute value of the difference in position of the 
values of the horizontal component of the block shift 

10 information, that correspond to the primary maximum and 

the secondary maximum of the first frequency 
distribution, is less than a predefined first difference 
threshold; 

f ^ - the absolute value of the difference in position of the 

1-53 values of the vertical component of the block shift 

: f information, that correspond to the primary maximum and 

the secondary maximum of the second frequency 
M : distribution, is less than a predefined second difference 

threshold; 

2;Q - the primary maximum of the first frequency distribution 

i ; I is greater than a first frequency threshold; 

- the primary maximum of the second frequency distribution 
is greater than a second frequency threshold. 

:1= The method according to alternative independent Claim 4 can 
25 thus be used to determine image shifts in an image sequence, 
thus reducing computing effort. 

A criterion that is suitable for detecting local movements 
within relevant image zone 6, 7, 8, and which therefore means 
that the reliability of the zone shift determination is 
30 inadequate, was given for both the method according to the 

main claim and the method according to alternative independent 
Claim 4 . 
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What is claimed is: 



1. A method for determining an image shift (15) in an image 
sequence, in particular to compensate for a camera movement, a 
plurality of image zones (6, 7, 8) of images being available 
for determining the shift (15); each image zone (6, 7, 8) 
being provided at a specific position in the images; each 
image zone (6, 7, 8) having predefined dimensions, in 
particular predefined numbers of pixels in different image 
directions , 

wherein, in order to correct a first image (14) in the image 
sequence, the shift (15) is determined either from image data 
of the first image (14) and from image data of a second image 
(12) that preferably directly precedes the first image (14) in 
the image sequence or from image data of the first image (14) 
and from image data of a second input image (11) that 
preferably directly precedes the first image (14) in the image 
sequence; a zone shift of an image zone (6, 7, 8) being used 
as the shift (15) as a function of a reliability for the zone 
shift determination of the one image zone (6, 7, 8); the zone 
shift of any given image zone (6, 7, 8) from the multiplicity 
of image zones (6, 7, 8) being determined either from the 
image data of the first image (14) and of the second image 
(12) within a given image zone (6, 7, 8) or from the image 
data of the first image (14) and of the second input image 
(11) within a given image zone (6, 7, 8) ; a method having the 
following steps being carried out to determine the zone shift 
in any two image zones (6, 7, 8) and to determine a 
reliability for the zone shift determination: 

the zone shift and a correlation quotient are formed for 

each of the two image zones (6, 7); 

- a threshold value function is determined as a function of 
the corresponding value of the determined zone shifts in the 
two image zones (6, 7) ; 

- the correlation quotient of the one of the two image zones 
(6, 7) is compared to the value obtained from the threshold 
value function for the zone shift of the other of the two 
image zones (6 7) ; 

- the reliability of the zone shift determination is deemed to 
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be adequate for one of the two image zones (6, 7) if the 
correlation quotient determined for the one image zone (6, 
7) is greater than the value of the threshold value function 
to be compared to it. 

2. The method according to Claim 1, 

wherein the correlation quotient for one of the plurality of 
image zones (6, 7 , 8) is determined according to a method 
having the following steps: 

- shift correlation values are determined for multiple 
possible zone shifts, using a block-matching method; 

- the zone shift for which maximum shift correlation values 
are achieved is deemed to be the zone shift of the image 
zone (6, 7, 8) ; 

- the correlation quotient is formed from the maximum of the 
shift correlation values, divided by the average for the 
determined shift correlation values. 

3. The method according to Claim 1, 

wherein, for any given value of a zone shift that is less than 
a predefined first threshold value, the threshold value 
function assumes the value of a predefined second threshold 
value; and for any given value of a zone shift that is greater 
than the predefined first threshold value, the threshold value 
function assumes the value of the predefined second threshold 
value minus a product, with the product including a predefined 
gradient parameter and a difference as factors; and the 
difference being formed from the given zone shift and the 
predefined first threshold value. 

4. A method for determining an image shift in a sequence of 
images, in particular to compensate for a camera movement, 
with at least one image zone (6, 7, 8) of images being 
available to determine the shift (15) ; the at least one image 
zone (6, 7, 8) being provided at a predefined position in the 
images; the at least one image zone (6, 7, 8) having 
predefined dimensions, in particular predefined numbers of 
pixels in different directions of the image, 

wherein, in order to correct a first image (14) in an image 
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sequence, the shift (15) is determined either from image data 
of the first image (14) and from image data of a second image 
(12) that preferably directly precedes the first image (14) in 
the image sequence or from image data of the first image (14) 
and from image data of a second input image (11) that 
preferably directly precedes the first image (14) in the image 
sequence; with a zone shift of the at least one image zone (6, 
7, 8) being used as the shift (15) ; block shift information, 
preferably shift vectors, from a block-based coding method 
being used for the at least one image zone (6, 7,8) to 
determine the zone shift; image blocks that are located in at 
least one image zone (6, 7, 8) being taken into account for 
the block shift information of the at least one image zone (6, 
7, 8) ; and the at least one image zone (6, 7, 8) being used to 
determine the shift (15) as a function of a reliability for 
the zone shift determination. 



5. The method according to Claim 4, 

wherein a method having the following steps is carried out for 
the at least one image zone (6, 7, 8) to determine the zone 
shift, which includes a horizontal and a vertical component, 
and to determine the reliability of the zone shift 
determination : 

- a first frequency distribution of the frequencies of 
different values for the horizontal component of the block 
shift information is established to determine the horizontal 
component of the zone shift, with the horizontal component 
of the zone shift corresponding to the value of the 
horizontal component of the block shift information for 
which the first frequency distribution assumes its primary 
maximum; 

- a second frequency distribution of the frequencies of 
different values for the vertical component of the block 
shift information is established to determine the vertical 
component of the zone shift, with the vertical component of 
the zone shift corresponding to the value of the vertical 
component of the block shift information for which the 
second frequency distribution assumes its primary maximum; 

- the reliability of the zone shift determination is deemed to 
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be adequate when all of the following conditions have been 
met : 

- the absolute value of the difference in position of the 
values corresponding to the primary maximum and the 
secondary maximum of the first frequency distribution of the 
horizontal component of the block shift information is less 
than a predefined first difference threshold; 

- the absolute value of the difference in position of the 
values corresponding to the primary maximum and the 
secondary maximum of the second frequency distribution of 
the vertical component of the block shift information is 
less than a predefined second difference threshold; 

- the primary maximum of the first frequency distribution is 
greater than a first frequency threshold; 

- the primary maximum of the second frequency distribution is 
greater than a second frequency threshold. 

6. The method according to one of the preceding claims, 
wherein an image movement, preferably produced by a camera 
movement, is separated from an additional movement that is 
superimposed on the image movement in some image zones (6, 7, 
8) of the image to be corrected, using the following steps: 

- the probability that the image movement will occur without 
the additional movement is determined at different positions 
of an image ; 

- the position and dimensions of a given image zone are 
determined and permanently specified as a function of the 
probability that the image movement will occur without the 
additional movement within the given image zone (6, 7, 8) ; 

- at least one first image zone (6, 7, 8) is preferably used 
to determine the shift (15) . 

7. The method according to one of the preceding claims, 
wherein the position and dimensions of the at least one first 
image zone (6, 7) are selected so that the at least one first 
image zone (6, 7) of the images to be corrected is largely 
filled by the image background. 

8. The method according to one of Claims 1 through 6, 
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wherein the position and dimensions of at least one second 
image zone (8) are selected so that the at least one second 
image zone (8) of the images to be corrected is largely filled 
with the image foreground. 

9. The method according to Claim 7 or 8, 

wherein both the at least one first image zone (6, 7) and the 
at least one second image zone (8) are available for 
determining a shift (15) . 

10. The method according to Claims 7 through 9, 

wherein two first image zones (6, 7) and a single second image 
zone (8) are available for correcting the image, with the 
shift (15) being determined in one of the following three ways 
in descending order of priority: 

- from the average of the zone shifts of the first two image 
zones (6 , 7), if the reliability of the zone shift 
determination of the first two image zones (6, 7) is deemed 
to be adequate; 

from the zone shift of the one of the two first image zones 
(6, 7) for which the reliability of the zone shift 
determination is deemed to be adequate; 

- from the zone shift of the second image zone (8) . 

11. The method according to Claim 10, 

wherein the method is used for a head- and- shoulder shot, with 
the first two image zones (6, 7) being selected in a side area 
to the left and right, preferably symmetrically to the 
vertical center axis of a predefined rectangular image; the 
distance of the first two image zones (6, 7) from the bottom 
of the image being greater than the distance of the first two 
image zones (6, 7) from the top of the image; the second image 
zone (8) being selected in the center of the image, preferably 
symmetrically to the vertical center axis of the rectangular 
image; the distance of the second image zone (8) from the top 
of the image being greater than the distance of the second 
image zone (8) from the bottom of the image. 

12. A device for determining a shift (15) using a method 
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according to one of the preceding claims, 

wherein the device includes a shift detecting circuit (100) 
and an enlarging circuit (200) , with the shift detecting 
circuit (100) having a zone shift detector (110) , an image 
storage device (12 0) , and a microcomputer (130) ; and the shift 
detecting circuit (100) determining the shift (15) . 
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(if only one name is listed below) or an original, first and 
joint inventor (if plural names are listed below) of the 
subject matter which is claimed and for which a patent is 
sought on the invention entitled METHOD AND DEVICE FOR 
DETERMINING AN IMAGE SHIFT IN AN IMAGE SEQUENCE, the 
specification of which was filed as PCT/DE00/00137 on January 
15, 2000. 

I hereby state that I have reviewed and understand 
the contents of the above-identified specification, including 
the claims. > 

I acknowledge the duty to disclose information which 
is material to the examination of this application in 
accordance with Title 37, Code of Federal Regulations, 
§ 1.56(a) . 

I hereby claim foreign priority benefits under Title 
35, United States Code, § 119 of any foreign application ( s ) 
for patent or inventor's certificate listed below and have 
also identified below any foreign application (s ) for patent or 
inventor's certificate having a filing date before that of the 
application on which priority is claimed: 
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with full power of substitution and revocation, to prosecute 
this application and to transact all business in the Patent 
and Trademark Office connected therewith. 
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KENYON & KENYON 
One Broadway 
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Mayer at (212) 425-7200. 

I hereby declare that all statements made herein of 
my own knowledge are true and that all statements made on 
information and belief are believed to be true; and further 
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willful false statements and the like so made are punishable 
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18 of the United States Code and that such willful and false 
statements may jeopardize the validity of the application or 
any patent issued thereon. 
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